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(57) ABSTRACT 

A mobile phone has a display with a touch screen. The 
device has a browser and is capable of retrieving a Web page 
from the Internet. The page is first displayed in its entirety. 
The user can recognize the page's general lay-out and 
presence of hyperlinks. When the user touches a particular 
location on the touch screen that corresponds to a portion of 
the page's image, the portion gets displayed so as to fill the 
display's area. Thus, the user can browse the Web with a 
display of limited size. 

6 Claims, 1 Drawing Sheet 
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HAND-HELD WITH AUTO-ZOOM FOR 
GRAPHICAL DISPLAY OF WEB PAGE 

RELATED APPLICATION 
This application is a continuation-in-part under 37 C.RR. 
§1.53(b).2 of co-pending U.S. Ser. No. 09/062,364 filed Apr. 
17, 1198 for Sung Choi and Jan van Ee for GRAPHICAL 
USER INTERFACE TOUCH SCREEN WITH AUTO 
ZOOM FEATURE. 

FIELD OF THE INVENTION 

The invention relates to graphical user interfaces (GUI), 
in particular to GUI's for devices with a relatively small 
screen real estate, such as handheld information appliances 
(palmtops, mobile phones, Web pads, PDA's or notebook 
computers, etc.) 

BACKGROUND ART 

Current trends indicate that by 2002 there will be IB 
subscribers worldwide to mobile phones. By 2004 there will 
be more Mobile phones in use than PC's. Mobile phones 
have become personal, trusted devices. Mobile phones, such 
a the Genie manufactured by Philips Electronics, typically 
have an LCD as part of the user interface and there is a trend 
to provide an ever larger number of onboard software 
services. 

Hand-held computing devices, such as personal digital 
assistants (PDA), e.g., the Palmlllx manufactured by 3COM 
or a Windows CE-based handheld, can be connected to the 
Internet via a wireless modem. As a result, ubiquitous 
information access via the Internet has started to become 
reality. Examples of a wireless modems are, for example, the 
Minstrel marketed by Novatel Wireless, and the Ricochet 
from Metricom. The Minstrel is a two-way wireless modem 
for a PDA that lets the user browse the Web and receive 
email, among other things. In a more general sense, a 
wireless modem like the Minstrel or Ricochet provides the 
handheld device with an IP address that can be used by any 
type of application that uses the Internet for communication 
(within limitations of throughput, latency and coverage). 
The Minstrel uses a technology referred to as Cellular 
Digital Packet Data (CDPD) that is supported by the cellular 
service providers. Web surfing is limited to a CDPD speed 
cap of 19.2-kbps. The Ricochet has a faster connect rate, in 
the 28.8K-bps range, but it is supported in only three 
metropolitan centers (the San Francisco Bay area, Seattle 
and Wash,). 

Studies further indicate that the functionalities of PDA's 
and mobile phones have started to converge, and that a 
mobile information society is developing. There will be an 
emerging of dedicated devices. PDA's are now work- 
related. In the near future PDA's will be personalized 
computers that stay with the user all the time. PDA's will get 
more power and smaller size and accommodate more, and 
more versatile, functionalities. 

Bandwidth and display size are believed to be the factors 
that limit the usability and practicality of the handheld 
device, be it a mobile phone, a palmtop or a hybrid. In 
particular, the GUI and the services accessible to such 
handhelds are critical factors for the consumers* acceptabil- 
ity of such services. In particular, e -commerce or electronic 
shopping may benefit from the ubiquity of handhelds if the 
implementing technology addresses the consumers' needs in 
terms of user-friendliness of the handheld devices. 

SUMMARY OF THE INVENTION 
Co-pending U.S. Ser. No. 09/062,364 (PHA 23,387) men- 
tioned above and incorporated herein by reference, corre- 
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sponds to published International Application WO9954807. 
This document relates to a graphical user interface touch 
screen for displaying controllable functions of an electronic 
device. The function is displayed as an icon and at a scale 

5 size in which the function is recognizable by a user but too 
small to easily access features of the function. A magnified 
version of at least an area of the icon is provided upon the 
user touching the area of the icon. For example, the device 
has a Virtual alphanumeric keyboard. The softkeys dis- 

10 played are too small for the user to select an individual one 
of them. Now, when the user touches the keyboard in a 
specific section that accommodates the desired key, the 
device's GUI magnifies that specific section so that the 
intended key re-appears at a larger scale and can be selected. 

15 The inventor has realized that this feature, referred to as 
"auto-zoom", is not only useful within the context of a 
user-interface for control functionalities represented graphi- 
cally by icons. The auto-zoom is also relevant to the 
rendering of any kind of graphical information on a display 

20 too small for the total information content, given the dis- 
play's resolution and size. For example, handheld informa- 
tion processing devices with Internet access (browsers) and 
displays, such as PDA's, palmtops, web pads, mobile 
phones using, e.g., the WAP (wireless application protocol) 

25 technology, etc., can be given browsers for retrieving and 
navigating web pages from the Internet, but they cannot 
render a page in its entirety without losing information. The 
lay-out and general appearance of the image of the page, 
however, indicate whether portions may or may not be of 

30 interest to the user, e.g., as containing hyperlinks. 

Such handheld devices provided with the auto -zoom 
feature let the user retrieve graphical information, e.g., a 
web page or streamed video that is stored, e.g., as a bitmap, 

^ in the display's framebuffer or another cache. 

Accordingly, the invention relates to an information pro- 
cessing apparatus that has an input for receiving data from 
an external resource, e.g. the Internet, a display, and a data 
processing system. The system is connected to the input and 

^ to the display. The system processes the data upon receipt 
and renders on the display an image corresponding to the 
data received. The apparatus has a touch screen for enabling 
a user to interact with the apparatus. The system is operative 
to enable the user to select via the touch screen a portion of 

45 the image when displayed at a first scale. Upon the portion 
being selected the system renders the selected portion on the 
display at a second scale larger than the first scale (zoom-in). 
The portion selected corresponds to a location on the touch 
screen. The invention thus allows the user to perceive the 

50 graphical information of the image regardless of the display 
size. The invention is especially interesting to handhelds, 
such as PDA's, palmtops,, mobile phones, web pads (thin 
clients with browsing capabilities), etc., because the size of 
a handheld 's display is necessarily small due to the required 

55 form factor and weight limitation. The ubiquitous informa- 
tion access via a browser is a great asset for Internet-enabled 
handhelds (comprising a wireless modem), as not only text 
pages but also, e.g., still pictures (e.g., jpeg), streaming 
video, web page with hyperlinks (e.g., HTML) and java 

6Q animation are now within reach of these devices whose 
screen real estate needs not be the limiting factor anymore. 

The apparatus in the invention can have one or more zoom 
levels. 

The term "texting" is currently being used to refer to the 
65 sending and receiving of short text messages using mobile 
phones. Many service providers allow users to key in and 
send short text messages (SMS: short messaging service). 
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Teenagers in Europe and East Asia seem to have embraced application for enabling the user to navigate the Web, and 

this technology, just as young people in the U.S. have gone the software for processing the graphical data as explained 

for live chat and instant messaging on the Internet. The herein. 

invention in U.S. Ser. No. 09/062,364 (PHA 23,387) When mc user has rct rieved a web page via modem 114 

improves the user-friendliness of the mobile phone's GUI. 5 the page gets displayed on LCD 102 in its entirety. Due to 

The invention of the auto-zoom extended to cover the the scale size of LCD 102 individual hyperlinks or text 

rendering of arbitrary graphical information further broad- fragments may not be well discernible, although the lay-out 

ens the scope of applicability of data processing handhelds, 0 f the pag6 conveys sufficient information to the user to 

especially in the field of consumer electronics (CE). determine what portion of the page may be relevant. Assume 

Screenphones may also benefit from the current inven- 10 that the portion is the upper right hand corner of the page, 

tion, A screenphone is typically a wired terminal that has a When the user now touches screen 104 in the associated 

small display monitor and a keyboard. The Internet screen- location or area, this action gets translated by processor 108 

phone market is expanding rapidly. Analysts predict rev- and under control of program memory 110 into a zooming-in 

enues in this market segment as over $2 B by 2002. The on that part of the page image that is centered around the 

success of the Minitel system in France has triggered the 15 touch location. This can be implemented, e.g., by a pre- 

introduction of screenphones elsewhere by banks to promote determined segmentation of the display area into, say, four 

home banking and shopping. areas that each can be selected for the zooming-in. 

Alternatively, that part of the original image gets magnified 

BRIEF DESCRIPTION OF THE DRAWING that has a center coinciding with the touch location 

The invention is explained below in further detail, by way 20 User-interaction with touch screen 104 causes display 102 

of example and with reference to the accompanying drawing to undergo a change in appearance. The change is preferably 

with a block diagram of a handheld device in the invention. effected through animation. Animation is the simulation of 

movement created by displaying a series of pictures, or 

DETAILED EMBODIMENTS frames, e.g., bitmaps. For example, an image of the original 

As specified above, the invention relates to, among other * P a f gradually develops in the zoomed in version of the area 

things, extending the autozoom keyboard idea in U.S. Ser. f lec 1 ted * the animation, the user perceives the 

No. 09/062,364 to any kind of graphical or displayable development from one image to the other as a continuous 

information, when the information cannot be displayed in its transition. The impression is created of a gradually changing 

entirety. For example, displaying a web page with hyper- 30 lay * 0ut Thus ' the aDimatlon av0lds the impression of an 

links on a mobile phone, which has a small LCD with touch abrupt confrontation with a new lay-out that requires the 

screen functionality, may render the user-interaction with the user to re -° rientate him/herself. The animation is effected 

links cumbersome if possible at all, due to their size. The ±r ° u S h pro P^ Processing of the bitmaps in memory 106 

LCD can nevertheless display the web page in its entirety by and frame buffer U2, 

scaling it down. This initial rendering upon retrieval of the 35 Incorporated by reference herein are the following: 
page from the Internet may very well cause the page U.S. Ser. No. 09/062,364 (attorney docket PHA 23,387) 
illegible. However, proper filtering of the scaled down image fi^d Apr. 17, 1998 for Sung Choi and Jan van Ee for 
ensures that the user can still recognize screen areas of GRAPHICAL USER INTERFACE TOUCH SCREEN 
potential interest and the hyperlinks therein. When the user WITH AUTO ZOOM FEATURE. This document relates to 
touches the screen, the portion of the image underneath the 40 4 graphical user interface touch screen for displaying con- 
touched location is enlarged and displayed so that hyperlinks trollable functions of an electronic device. The function is 
can be individually be selected, possibly after a next, similar displayed as an icon and at a scale size in which the function 
zoom-in process. is recognizable by a user but too small to easily access 
FIG. 1 is a block diagram with main components of an features of the function. A magnified version of at least an 
apparatus 100 in the invention. Apparatus 100 is, for 45 area of the icon is provided upon the user touching the area 
example, a mobile phone or a palmtop PC with Internet °^ tne * COD * 

access. Apparatus 100 comprises a display 102 for display of U.S. Ser. No. 09/128,839 (attorney docket PHA 23,469) 

a graphical information, and a touch screen 104 for user- filed Aug. 4, 1998 for Jan van Ee for REMOTE CONTROL 

interaction with the apparatus. Display 102 comprises, e.g., HAS ANIMATED GUI. This document relates to a remote 

an LCD. Touchscreen 104 is, for example, a resistive tablet. 50 control device for remote control of equipment such as a 

For more background on such input devices, see, for home theater. The remote has a display for display of a GUI 

example, U.S. Pat. Nos. 5,402,151; 5,231,381; 5,777,607 that enables a user to interact with the device. User- 

and 5,767,458 of Philips Electronics, all incorporated herein interaction with the device causes the GUI to undergo a 

by reference. LCD 102 and touch screen 104 are physically change in appearance. The change is effected through ani- 

iotegrated. Apparatus 100 comprises a random-access 55 mation. Animation is the simulation of movement created by 

memory 106, a microprocessor 108, and a program memory displaying a series of pictures, or frames, e.g., bitmaps. For 

110 (e., an EEPROM). A portion of memory 106 comprises example, a panel with clustered control options slides out of 

a frame buffer 112. Frame buffer 112 is coupled to display y iew and a next one slides into view, or displayed icons slide 

102 and stores the information content shown on display to new positions while new icons appear, etc. Through the 

102. Memory 106 stores bitmaps that are mapped onto 60 animation, the user perceives the development from one 

display 102 via frame buffer 106 under control of a software panel to the other as a continuous transition. The impression 

bit-blitter run on microprocessor 108. Microprocessor 108 is created of a gradually changing lay-out, of scrolling 

receives user-input via touch screen 104 and translates the panels, of sliding, rotating, expanding or contracting icons, 

input into associated GUI actions via frame buffer 112 under etc. Thus, the animation avoids the impression of an abrupt 

control of the program memory U0. Handheld 100 com- 65 confrontation with a new lay-out. 

prises a wireless modem 114 for connecting to Internet 116. U.S. Ser. No. 09/427,821 (attorney docket PHA 23,786) 

Program memory 110 stores, among other things, a browser filed Oct. 27, 1999 for Joost Kemink and Richard Sagar for 
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PDA HAS WIRELESS MODEM FOR REMOTE CON- 
TROL VIA THE INTERNET. This document relates to a 
PDA combined with a wireless modem to enable remote 
control of CE equipment via the Internet and a local home 
server. More specifically, The wireless modem enables com- 5 
munication with a server via a data network such as the 
Internet. A control network is coupled between the server 
and controllable equipment. The handheld^ now capable of 
functibnihg' as a wireless remote control device for the 
equipment via the Internet and the server. The system may 10 
comprise a video camera together with hardware and soft- 
ware to create a formatted still image suitable for being 
displayed on the handheld device. The user can now instruct 
retrieval of a still image from the server via the Internet. This 
application serves as, e.g., a security system that enables the 15 
remote user to monitor his/her front porch, or to monitor a 
child by way of a remote (or fall-back) baby-sit. The 
user-accessibility of equipment is guaranteed by the ubiquity 
of the Internet, thus enabling to expand the range of control 
and monitoring capabilities for a mobile user. 20 
What is claimed is: 

1. A handheld communication device comprising: 

a wireless modem for receiving data; 

a display that has a substantially small size suitable for in 

the handheld communication device; 25 
a data processing system connected to the modem and to 

the display for processing the received data and for 

rendering an image corresponding to the data received; 
a touch screen for enabling a user to interact with the 30 

device; 
wherein: 

the system is operative to enable the user to select through 
a touch location on the touch screen a portion of the 
image, when displayed at a first scale, for rendering the 
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selected portion on the display at a second scale larger 
than the first scale thereby facilitating a selection of a 
feature; and 

the selected portion when rendered at the first scale is a 
zoomed- in version of part of the image at the first scale 
substantially centered around the touch screen. 

2. The device of claim 1, wherein a position of the touch 
location is arbitrary with respect to" the touch screen. 

3. The device of claim 1, comprising a browser. 

4. The device of claim 1, having wireless Internet access, 

5. The device of claim 1, wherein the data comprises 
streaming video. 

6. Software for being installed on a handheld communi- 
cation device comprising: 

a wireless modem for receiving data; 

a display that has a substantially small size suitable for in 
the handheld communication device; 

a data processing system connected to the input and to the 
display for processing the received data and for ren- 
dering an image corresponding to the data received; 

a touch screen for enabling a user to interact with the 
device; 
wherein: 

the software is operative to enable the user to select 
through a touch location on the touch screen a portion 
of the image, when displayed at a first scale, for 
rendering the selected portion on the display at a 
second scale larger than the "first scale;" and, thereby 
facilitating a selection of a feature, 

the selected portion when rendered at the first scale is a 
zoomed-in version of part of the image at the first scale 
substantially centered around the touch screen. 
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